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1.  Introduction 


1.1.  Research  Objectives 

As  part  of  this  research  program  we  proposed  the  development  of  an  energy-based  thermo¬ 
dynamic  stabilization  framework  for  hybrid  control  design  of  large-scale  aerospace  systems. 
In  particular,  we  concentrate  on  hybrid  control,  hierarchical  control,  impulsive  dynamical 
systems,  nonnegative  dynamical  systems,  compart  mental  systems,  large-scale  systems,  non¬ 
linear  switching  control,  cooperative  control,  and  adaptive  control.  Application  areas  include 
large  flexible  interconnected  space  structures,  spacecraft  stabilization,  cooperative  control  of 
unmanned  air  vehicles,  network  systems,  swarms  of  air  and  space  vehicle  formations,  and 
pharmacological  systems. 

1.2.  Overview  of  Research 

Controls  research  by  the  Principal  Investigator  1-74]  has  concentrated  on  an  energy- 
based  thermodynamic  stabilization  framework  for  hybrid  control  design  of  large-scale  aero¬ 
space  systems.  This  framework  provides  a  rigorous  foundation  for  developing  a  unified 
energy-based  (network  thermodynamic)  analysis  and  synthesis  methodology  for  large-scale 
aerospace  systems  possessing  hybrid,  hierarchical,  and  feedback  structures.  This  framework 
additionally  provides  a  rigorous  alternative  to  designing  gain  scheduled  controllers  for  gen¬ 
eral  nonlinear  dynamical  systems  by  constructing  minimal  complexity  logic-based  nonlinear 
( ontrollers  consisting  of  a  number  of  subcontrollers  situated  in  levels  (protocol  layers  of  hier¬ 
archies)  such  that  each  subcontroller  can  coordinate  lower-level  controllers.  Correspondingly, 
one  of  the  main  goal  of  this  research  has  been  to  make  progress  towards  the  development 
of  analysis  and  hierarchical  hybrid  nonlinear  control  law  tools  for  nonlinear  large-scale  dy¬ 
namical  systems.  This  framework  provides  the  basis  for  developing  control-system  parti¬ 
tioning/embedding  using  concepts  of  energy-based  thermodynamic  hybrid  stabilization  for 
complex,  large-scale  aerospace  systems. 

A  thermodynamic  stabilization  framework  for  Eulerian  swarm  models  is  also  developed. 
Specifically,  we  present  a  distributed  boundary  controller  architecture  involving  the  exchange 
of  information  between  uniformly  distributed  swarms  that  guarantee  that  the  closed-loop 
system  is  consistent  with  basic  thermodynamic  principles.  Robustness  of  individual  agent 
failures  and  unplanned  individual  agent  behavior  is  also  addressed.  In  addition,  a  general 
framework  for  designing  semistable  protocols  in  dynamical  networks  with  switching  topolo¬ 
gies  is  also  developed.  Specifically,  we  develop  a  distributed  nonlinear  controller  architecture 
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for  inultiagent  network  consensus  with  time-dependent  and  state-dependent  communication 
topologies.  Correspondingly,  the  main  goal  of  this  research  over  the  past  year  has  been  to 
make  progress  towards  the  development  of  analysis  and  control  for  nonlinear  multi-agent 
systems. 

1.3.  Goals  of  this  Report 

The  main  goal  of  this  report  is  to  summarize  the  progress  achieved  under  the  program 
during  the  past  three  years.  Since  most  of  the  technical  results  appeared  or  will  soon  appear 
in  over  74  archival  journal  and  conference  publications,  we  shall  only  summarize  these  results 
and  remark  on  their  significance  and  interrelationship. 

2.  Description  of  Work  Accomplished 

The  following  partial  research  accomplishments  have  been  completed  over  the  past  three 
years. 

2.1.  Hybrid  Decentralized  Maximum  Entropy  Control  for  Large- 
Scale  Dynamical  Systems 

Modern  complex  dynamical  systems1  are  highly  interconnected  and  mutually  interdepen¬ 
dent,  both  physically  and  through  a  multitude  of  information  and  communication  network 
constraints.  The  sheer  size  (i.e.,  dimensionality)  and  complexity  of  these  large-scale  dy¬ 
namical  systems  often  necessitates  a  decentralized  architecture  for  analyzing  and  controlling 
these  systems.  Specifically,  in  the  control-system  design  of  complex  large-scale  dynamical 
systems  it  is  often  desirable  to  treat  the  overall  system  as  a  collection  of  interconnected 
subsystems.  The  behavior  of  the  composite  (i.e.,  large-scale)  system  can  then  be  predicted 
from  the  behaviors  of  the  individual  subsystems  and  their  interconnections.  The  need  for 
decentralized  control  design  of  large-scale  systems  is  a  direct  consequence  of  the  physical  size 
and  complexity  of  the  dynamical  model.  In  particular,  computational  complexity  may  be 
too  large  for  model  analysis  while  severe  constraints  on  communication  links  between  system 
sensors,  actuators,  and  processors  may  render  centralized  control  architectures  impractical. 
Moreover,  even  when  communication  constraints  do  not  exist,  decentralized  processing  may 
be  more  economical. 

!Here  we  have  in  mind  large  flexible  space  structures,  aerospace  systems,  electric  power  systems,  network 
systems,  economic  systems,  and  ecological  systems,  to  cite  but  a  few  examples. 
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The  complexity  of  modern  controlled  large-scale  dynamical  systems  is  further  exacer¬ 
bated  by  the  use  of  hierarchial  embedded  control  subsystems  within  the  feedback  control 
system;  that  is,  abstract  decision-making  units  performing  logical  checks  that  identity  sys¬ 
tem  mode  operation  and  specify  the  continuous- variable  subcontroller  to  be  activated.  Such 
systems  typically  possess  a  multiechelon  hierarchical  hybrid  decentralized  control  architec¬ 
ture  characterized  by  continuous-time  dynamics  at  the  lower  levels  of  the  hierarchy  and 
discrete-time  dynamics  at  the  higher  levels  of  the  hierarchy.  The  lower-level  units  directly 
interact  with  the  dynamical  system  to  be  controlled  while  the  higher-level  units  receive  infor¬ 
mation  from  the  lower-level  units  as  inputs  and  provide  (possibly  discrete)  output  commands 
which  serve  to  coordinate  and  reconcile  the  (sometimes  competing)  actions  of  the  lower-level 
units.  The  hierarchical  controller  organization  reduces  processor  cost  and  controller  com¬ 
plexity  by  breaking  up  the  processing  task  into  relatively  small  pieces  and  decomposing  the 
fast  and  slow  control  functions.  Typically,  the  higher-level  units  perform  logical  checks  that 
determine  system  mode  operation,  while  the  lower-level  units  execute  continuous-variable 
commands  for  a  given  system  mode  of  operation. 

Since  implementation  constraints,  cost,  and  reliability  considerations  often  require  de¬ 
centralized  controller  architectures  for  controlling  large-scale  systems,  decentralized  control 
has  received  considerable  attention  in  the  literature.  A  straightforward  decentralized  control 
design  technique  is  that  of  sequential  optimization ,  wherein  a  sequential  centralized  subcon¬ 
troller  design  procedure  is  applied  to  an  augmented  closed-loop  plant  composed  of  the  actual 
plant  and  the  remaining  subcontrollers.  Clearly,  a  key  difficulty  with  decentralized  control 
predicated  on  sequential  optimization  is  that  of  dimensionality.  An  alternative  approach  to 
sequential  optimization  for  decentralized  control  is  based  on  subsystem  decomposition  with 
centralized  design  procedures  applied  to  the  individual  subsystems  of  the  large-scale  system. 
Decomposition  techniques  exploit  subsystem  interconnection  data  and  in  many  cases,  such 
as  in  the  presence  of  very  high  system  dimensionality,  is  absolutely  essential  for  designing 
decentralized  controllers. 

In  this  research  [6],  we  develop  a  novel  energy-based  hybrid  decentralized  control  frame¬ 
work  for  lossless  and  dissipative  large-scale  dynamical  systems  based  on  subsystem  decompo¬ 
sition.  The  notion  of  energy  here  refers  to  abstract  energy  notions  for  which  a  physical  system 
energy  interpretation  is  not  necessary.  These  dynamical  systems  cover  a  very  broad  spec¬ 
trum  of  applications  including  aerospace  systems,  fluid  systems,  electromechanical  systems, 
electrical  systems,  combustion  systems,  structural  vibration  systems,  biological  systems, 
physiological  systems,  power  systems,  telecommunications  systems,  and  economic  systems, 
to  cite  but  a  few  examples.  The  concept  of  an  energy-based  hybrid  decentralized  controller 
can  be  viewed  as  a  feedback  control  technique  that  exploits  the  coupling  between  a  physi- 
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cal  large-scale  dynamical  system  and  an  energy-based  decentralized  controller  to  efficiently 
remove  energy  from  the  physical  large-scale  system.  Specifically,  if  a  dissipative  or  lossless 
large-scale  system  is  at  high  energy  level,  and  a  lossless  feedback  decentralized  controller 
at  a  low  energy  level  is  attached  to  it,  then  subsystem  energy  will  generally  tend  to  flow 
from  each  subsystem  into  the  corresponding  subcontroller,  decreasing  the  subsystem  energy 
and  increasing  the  subcontroller  energy  [1].  Of  course,  emulated  energy,  and  not  physical 
energy,  is  accumulated  by  each  subcontroller.  Conversely,  if  each  attached  subcontroller  is 
at  a  high  energy  level  and  the  corresponding  subsystem  is  at  a  low  energy  level,  then  energy 
can  flow  from  each  subcontroller  to  each  corresponding  subsystem,  since  each  subcontroller 
can  generate  real,  physical  energy  to  effect  the  required  energy  flow.  Hence,  if  and  when 
the  subcontroller  states  coincide  with  a  high  emulated  energy  level,  then  we  can  reset  these 
states  to  remove  the  emulated  energy  so  that  the  emulated  energy  is  not  returned  to  the 
plant.  In  this  case,  the  overall  closed-loop  system  consisting  of  the  plant  and  the  controller 
possesses  discontinuous  flows  since  it  combines  logical  switchings  with  continuous  dynamics, 
leading  to  impulsive  differential  equations  [l]. 

2.2.  Control  Vector  Lyapunov  Functions  for  Large-Scale  Hybrid 

Dynamical  Systems 

In  this  research  [5],  we  provide  generalizations  to  the  recent  extensions  of  vector  Lyapunov 
theory  for  continuous-time  systems  [2]  to  address  stability  and  control  design  of  impulsive 
dynamical  systems  via  vector  Lyapunov  functions.  Vector  Lyapunov  theory  has  been  devel¬ 
oped  to  weaken  the  hypothesis  of  standard  Lyapunov  theory  in  order  to  enlarge  the  class 
of  Lyapunov  functions  that  can  be  used  for  analyzing  system  stability.  Lyapunov  methods 
have  also  been  used  by  control  system  designers  to  obtain  stabilizing  feedback  controllers 
for  nonlinear  systems.  In  particular,  for  smooth  feedback,  Lyapunov-based  methods  were 
inspired  by  Jurdjevic  and  Quinn  who  give  sufficient  conditions  for  smooth  stabilization  based 
on  the  ability  of  constructing  a  Lyapunov  function  for  the  closed-loop  system.  More  recently, 
Artstein  introduced  the  notion  of  a  control  Lyapunov  function  whose  existence  guarantees  a 
feedback  control  law  which  globally  stabilizes  a  nonlinear  dynamical  system.  Even  though 
for  certain  classes  of  nonlinear  dynamical  systems  a  universal  construction  of  a  feedback 
stabilizer  can  be  obtained  using  control  Lyapunov  functions,  there  does  not  exist  a  uni¬ 
fied  procedure  for  finding  a  Lyapunov  function  candidate  that  will  stabilize  the  closed-loop 
system  for  general  nonlinear  systems. 

In  an  attempt  to  simplify  the  construction  of  Lyapunov  functions  for  the  analysis  and 
control  design  of  nonlinear  dynamical  systems,  several  researchers  have  resorted  to  vector 
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Lyapunov  functions  as  an  alternative  to  scalar  Lyapunov  functions.  Vector  Lyapunov  func¬ 
tions  were  first  introduced  by  Bellman  and  Matrosov,  and  are  ideal  for  analyzing  large-scale 
systems  [1  2,5].  The  use  of  vector  Lyapunov  functions  in  dynamical  system  theory  offers  a 
very  flexible  framework  since  each  component  of  the  vector  Lyapunov  function  can  satisfy 
less  rigid  requirements  as  compared  to  a  single  scalar  Lyapunov  function.  Weakening  the 
hypothesis  on  the  Lyapunov  function  enlarges  the  class  of  Lyapunov  functions  that  can  be 
used  for  analyzing  system  stability.  In  particular,  each  component  of  a  vector  Lyapunov 
function  need  not  be  positive  definite  with  a  negative  or  even  negative-semidefinite  deriva¬ 
tive.  Alternatively,  the  time  derivative  of  the  vector  Lyapunov  function  need  only  satisfy  an 
element-by-element  inequality  involving  a  vector  field  of  a  certain  comparison  system.  Since 
in  this  case  the  stability  properties  of  the  comparison  system  imply  the  stability  properties 
of  the  dynamical  system,  the  use  of  vector  Lyapunov  theory  can  significantly  reduce  the 
complexity  (i.e.,  dimensionality)  of  the  dynamical  system  being  analyzed. 

The  results  of  this  research  [5]  build  on  those  of  [2]  and  include  a  generalized  compari¬ 
son  principle  involving  hybrid  comparison  dynamics  that  are  dependent  on  the  comparison 
system  states  as  well  as  the  nonlinear  impulsive  dynamical  system  states.  Next,  we  develop 
stability  theorems  based  on  hybrid  comparison  inequalities  as  well  as  partial  stability  re¬ 
sults  for  impulsive  systems  using  vector  Lyapunov  functions.  Furthermore,  we  extend  the 
newly  developed  notion  of  control  vector  Lyapunov  functions  presented  in  [2]  to  impulsive 
dynamical  systems  and  show  that  in  the  case  of  a  scalar  comparison  system  the  definition 
of  a  control  vector  Lyapunov  function  collapses  into  a  combination  of  the  classical  definition 
of  a  control  Lyapunov  function  for  continuous-time  dynamical  systems  and  the  definition  of 
a  control  Lyapunov  function  for  discrete-time  dynamical  systems.  In  addition,  using  control 
vector  Lyapunov  functions,  we  present  a  universal  hybrid  decentralized  feedback  stabilizer 
for  a  decentralized  affine  in  the  control  nonlinear  impulsive  dynamical  system  with  guaran¬ 
teed  gain  and  sector  margins.  These  results  are  then  used  to  develop  hybrid  decentralized 
controllers  for  large-scale  impulsive  dynamical  systems  with  robustness  guaiantees  against 
full  modeling  and  input  uncertainty. 

2.3.  Consensus  and  Semistability  in  Network  Dynamical  Systems 
with  Arbitrary  Time-Delays 

As  discussed  in  Section  2.1,  modern  complex  dynamical  systems  are  highly  interconnected 
and  mutually  interdependent,  both  physically  and  through  a  multitude  of  information  and 
communication  networks.  By  properly  formulating  these  systems  in  terms  of  subsystem  in¬ 
teraction  involving  energy/mass  transfer,  the  dynamical  models  of  many  of  these  systems 
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can  be  derived  from  mass,  energy,  and  information  balance  considerations  that  involve  dy¬ 
namic  states  whose  values  are  nonnegative.  Hence,  it  follows  from  physical  considerations 
that  the  state  trajectory  of  such  systems  remains  in  the  nonnegative  orthant  of  the  state 
space  for  nonnegative  initial  conditions.  Such  systems  are  commonly  referred  to  as  nonneg- 
ative  dynamical  systems  in  the  literature.  A  subclass  of  nonnegative  dynamical  systems  are 
compartment  al  systems .  Compart  mental  systems  involve  dynamical  models  that  are  char¬ 
acterized  by  conservation  laws  (e.g.,  mass  and  energy)  capturing  the  exchange  of  material 
between  coupled  macroscopic  subsystems  known  as  compartments.  Each  compartment  is 
assumed  to  be  kinetically  homogeneous,  that  is,  any  material  entering  the  compartment  is 
instantaneously  mixed  with  the  material  of  the  compartment.  The  range  of  applications 
of  nonnegative  systems  and  cornpartmental  systems  includes  biological  and  physiological 
systems,  chemical  reaction  systems,  queuing  systems,  large-scale  systems,  stochastic  sys¬ 
tems  (whose  state  variables  represent  probabilities),  ecological  systems,  economic  systems, 
demographic  systems,  telecommunications  systems,  transportation  systems,  power  systems, 
thermodynamic  systems,  arid  structural  vibration  systems,  to  cite  but  a  few  examples. 

A  key  physical  limitation  of  cornpartmental  systems  is  that  transfers  between  compart¬ 
ments  are  not  instantaneous  and  realistic  models  for  capturing  the  dynamics  of  such  systems 
should  account  for  material,  energy,  or  information  in  transit  between  compartments.  Hence, 
to  accurately  describe  the  evolution  of  the  aforementioned  systems,  it  is  necessary  to  include 
in  any  mathematical  model  of  the  system  dynamics  some  information  of  the  past  system 
states.  In  this  case,  the  state  of  the  system  at  a  given  time  involves  a  piece  of  trajectories 
in  the  space  of  continuous  functions  defined  on  an  interval  in  the  nonnegative  orthant  of  the 
state  space.  This  of  course  leads  to  (infinite-dimensional)  delay  dynamical  systems. 

Nonnegative  and  cornpartmental  models  are  also  widespread  in  agreement  problems  in 
networks  with  directed  graphs  and  switching  topologies.  Specifically,  distributed  decision¬ 
making  for  coordination  of  networks  of  dynamic  agents  involving  information  flow  can  be 
naturally  captured  by  compart  mental  models.  These  dynamical  network  systems  cover  a  very 
broad  spectrum  of  applications  including  cooperative  control  of  unmanned  air  vehicles,  dis¬ 
tributed  sensor  networks,  swarms  of  air  and  space  vehicle  formations,  and  congestion  control 
in  communication  networks.  In  many  applications  involving  multiagent  systems,  groups  of 
agents  are  required  to  agree  oil  certain  quantities  of  interest.  In  particular,  it  is  important  to 
develop  consensus  protocols  for  networks  of  dynamic  agents  with  directed  information  flow, 
switching  network  topologies,  and  possible  system  time-delays.  In  this  research  [13],  we  use 
compartrnental  dynamical  system  models  to  characterize  dynamic  algorithms  for  linear  and 
nonlinear  networks  of  dynamic  agents  in  the  presence  of  inter-agent  communication  delays 
that  possess  a  continuum  of  semistable  equilibria,  that  is,  protocol  algorithms  that  guaran- 
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tee  convergence  to  Lyapunov  stable  equilibria.  In  addition,  we  show  that  the  steady-state 
distribution  of  the  dynamic  network  is  uniform,  leading  to  system  state  equipartitioning  or 
consensus.  These  results  extend  the  results  in  the  literature  on  consensus  protocols  for  linear 
balanced  networks  to  linear  and  nonlinear  unbalanced  networks  with  switching  topologies 
and  time-delays. 

2.4.  Finite-Time  Semistable  Consensus  Protocols  for  Dynamical 
Networks 

In  recent  research  [19]  a  unified  stability  analysis  framework  for  systems  having  a  con¬ 
tinuum  of  equilibria  was  developed.  Since  every  neighborhood  of  a  nonisolated  equilibrium 
contains  another  equilibrium,  a  nonisolated  equilibrium  cannot  be  asymptotically  stable. 
Hence,  asymptotic  stability  is  not  the  appropriate  notion  of  stability  for  systems  having  a 
continuum  of  equilibria.  Two  notions  that  are  of  particular  relevance  to  such  systems  are 
convergence  and  semistability.  Convergence  is  the  property  whereby  every  system  solution 
converges  to  a  limit  point  that  may  depend  on  the  system  initial  condition.  Semistability 
is  the  additional  requirement  that  all  solutions  converge  to  limit  points  that  are  Lyapunov 
stable.  Sernistability  thus  implies  Lyapunov  stability,  and  is  implied  by  asymptotic  stability. 

The  dependence  of  the  limiting  state  on  the  initial  state  is  seen  in  numerous  dynamical 
systems  including  cornpartinental  systems  which  arise  in  chemical  kinetics,  and  biomedical, 
environmental,  economic,  power,  and  thermodynamic  systems  10].  For  these  systems,  every 
trajectory  that  starts  in  a  neighborhood  of  a  Lyapunov  stable  equilibrium  converges  to  a 
(possibly  different)  Lyapunov  stable  equilibrium,  and  hcncc,  these  systems  are  semistable.  In 
addition  to  semistability,  it  is  desirable  that  a  dynamical  system  that  exhibits  sernistability 
also  possesses  the  property  that  trajectories  that  converge  to  a  Lyapunov  stable  system  state 
must  do  so  in  finite  time  rather  than  merely  asymptotically. 

In  this  research  [19],  we  merge  the  theories  of  sernistability  and  finite-time  stability  to 
develop  a  rigorous  framework  for  finite-time  sernistability.  In  particular,  finite-time  semista- 
bility  for  a  continuum  of  equilibria  of  continuous  autonomous  systems  is  established.  Con¬ 
tinuity  of  the  settling-time  function  as  well  as  Lyapunov  and  converse  Lyapunov  theorems 
for  sernistability  are  also  developed.  In  addition,  necessary  and  sufficient  conditions  for 
finite-time  sernistability  of  homogeneous  systems  are  addressed  by  exploiting  the  fact  that  a 
homogeneous  system  is  finite-time  seinistable  if  and  only  if  it  is  semistable  and  has  a  negative 
degree  of  homogeneity. 

Next,  we  use  these  results  to  develop  a  general  framework  for  designing  semistable  pro¬ 
tocols  in  dynamical  networks  for  achieving  coordination  tasks  in  finite  time.  Distributed 
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decision-making  for  coordination  of  networks  of  dynamic  agents  involving  information  flow 
can  be  naturally  captured  by  graph-theoretic  notions.  These  dynamical  network  systems 
cover  a  very  broad  spectrum  of  applications  including  cooperative  control  of  unmanned  air 
vehicles  (UAV’s),  autonomous  underwater  vehicles  (AUV’s),  distributed  sensor  networks, 
air  and  ground  transportation  systems,  swarms  of  air  and  space  vehicle  formations,  and 
congestion  control  in  communication  networks,  to  cite  but  a  few  examples.  Hence,  it  is 
not  surprising  that  a  considerable  research  effort  has  been  devoted  to  control  of  networks 
and  control  over  networks  in  recent  years.  However,  finite-time  coordination  has  not  been 
addressed  in  the  literature. 

In  many  applications  involving  multiagent  systems,  groups  of  agents  are  required  to  agree 
on  certain  quantities  of  interest.  In  particular,  it  is  important  to  develop  information  con¬ 
sensus  protocols  for  networks  of  dynamic  agents  wherein  a  unique  feature  of  the  closed-loop 
dynamics  under  any  control  algorithm  that  achieves  consensus  in  a  dynamical  network  is  the 
existence  of  a  continuum  of  equilibria  representing  a  state  of  consensus.  Under  such  dynam¬ 
ics,  the  limiting  consensus  state  achieved  is  not  determined  completely  by  the  dynamics,  but 
depends  on  the  initial  system  state.  Hence,  using  the  results  on  finite-time  semistability,  we 
develop  a  unified  framework  for  addressing  the  consensus  problem  in  networks  of  agents  in 
finite  time.  Specifically,  we  develop  nonlinear  finite-time  controllers  using  undirected  and 
directed  graphs  to  accommodate  for  a  full  range  of  possible  graph  information  topologies 
without  limitations  of  bidirectional  communication. 

2.5.  Continuous  and  Hybrid  Distributed  Control  for  Multiagent 
Coordination:  Consensus,  Flocking,  and  Cyclic  Pursuit 

Modern  complex  dynamical  systems  are  highly  interconnected  and  mutually  interdepen¬ 
dent,  both  physically  and  through  a  multitude  of  information  and  communication  networks. 
Distributed  decision-making  for  coordination  of  networks  of  dynamic  agents  involving  in¬ 
formation  flow  can  be  naturally  captured  by  graph-theoretic  notions.  These  dynamical 
network  systems  cover  a  very  broad  spectrum  of  applications  including  cooperative  control 
of  unmanned  air  vehicles  (UAV’s),  autonomous  underwater  vehicles  (AUV’s),  distributed 
sensor  networks,  air  and  ground  transportation  systems,  swarms  of  air  and  space  vehicle 
formations,  and  congestion  control  in  communication  networks,  to  cite  but  a  few  examples. 

A  key  application  area  within  aerospace  systems  is  cooperative  control  of  vehicle  forma¬ 
tions  using  distributed  and  decentralized  controller  architectures.  Distributed  control  refers 
to  a  control  architecture  wherein  the  control  is  distributed  via  multiple  computational  units 
that  are  interconnected  through  information  and  communication  networks,  whereas  decen- 


8 


tralized  control  refers  to  a  control  architecture  wherein  local  decisions  are  based  only  on  local 
information.  Vehicle  formations  are  typically  dynamically  decoupled,  that  is,  the  motion  of 
a  given  agent  or  vehicle  does  not  directly  affect  the  motion  of  the  other  agents  or  vehicles. 
The  multiagent  system  is  coupled  via  the  task  which  the  agents  or  vehicles  are  required  to 
perform. 

In  many  applications  involving  multiagent  systems,  groups  of  agents  are  required  to  agree 
on  certain  quantities  of  interest.  In  particular,  it  is  important  to  develop  information  con¬ 
sensus  protocols  for  networks  of  dynamic  agents  wherein  a  unique  feature  of  the  closed-loop 
dynamics  under  any  control  algorithm  that  achieves  consensus  is  the  existence  of  a  continuum 
of  equilibria  representing  a  state  of  equipartitioning  or  consensus.  Under  such  dynamics,  the 
limiting  consensus  state  achieved  is  not  determined  completely  by  the  dynamics,  but  depends 
on  the  initial  system  state  as  well.  As  discussed  in  Section  2.4,  for  such  systems  possessing 
a  continuum  of  equilibria,  sernist ability,  and  not  asymptotic  stability,  is  the  relevant  notion 
of  stability.  Semist ability  is  the  property  whereby  every  trajectory  that  starts  in  a  neighbor¬ 
hood  of  a  Lyapunov  stable  equilibrium  converges  to  a  (possibly  different)  Lyapunov  stable 
equilibrium. 

Alternatively,  in  other  applications  of  multiagent  systems,  groups  of  agents  are  required  to 
achieve  and  maintain  a  prescribed  geometric  shape.  This  formation  problem  includes  flocking 
and  cyclic  pursuit ,  wherein  parallel  and  circular  formations  of  vehicles  are  sought.  For 
formation  control  of  multiple  vehicles,  cohesion,  separation,  and  alignment  constraints  are 
typically  required  for  individual  agent  steering  which  describe  how  a  given  vehicle  maneuvers 
based  on  the  positions  and  velocities  of  nearby  agents.  Specifically,  cohesion  refers  to  a 
steering  rule  wherein  a  given  vehicle  attempts  to  move  toward  the  average  position  of  local 
vehicles,  separation  refers  to  collision  avoidance  with  nearby  vehicles,  while  alignment  refers 
to  velocity  matching  with  nearby  vehicles. 

Using  graph-theoretic  notions,  in  this  research  17,58]  we  develop  a  unified  framework 
for  addressing  consensus,  flocking,  and  cyclic  pursuit  problems  for  multiagent  dynamical 
systems.  Specifically,  we  present  continuous  and  hybrid  distributed  and  decentralized  con¬ 
troller  architectures  for  multiagent  coordination.  In  contrast  to  virtually  all  of  the  existing 
results  in  the  literature  on  control  of  networks,  the  majority  of  the  proposed  controllers  are 
dynamic  compensators.  The  proposed  controller  architectures  are  predicated  on  the  recently 
developed  notion  of  system  thermodynamics  [10]  resulting  in  thermodynamically  consistent 
continuous  and  hybrid  controller  architectures  involving  the  exchange  of  information  be¬ 
tween  agents  that  guarantee  that  the  closed-loop  dynamical  network  is  consistent  with  basic 
thermodynamic  principles.  Another  unique  feature  of  our  framework  is  that  several  of  the 
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proposed  controller  architectures  are  hybrid,  and  hence,  the  overall  closed-loop  dynamics 
under  these  controller  algorithms  achieving  consensus,  flocking,  or  cyclic  pursuit  possesses 
discontinuous  flows  since  they  combine  logical  switchings  with  continuous  dynamics,  leading 
to  impulsive  differential  equations  [1].  The  proposed  controllers  use  undirected  and  directed 
graphs  to  accommodate  for  a  full  range  of  possible  graph  information  topologies  without 
limitations  of  bidirectional  communication. 

2.6.  Robust  Control  Algorithms  for  Network  Consensus  Protocols 
with  Uncertain  Communication  Graph  Topologies 

Due  to  advances  in  embedded  computational  resources  over  the  last  several  years,  a 
considerable  research  effort  has  been  devoted  to  the  control  of  networks  and  control  over 
networks.  Network  systems  involve  distributed  decision-making  for  coordination  of  networks 
of  dynamic  agents  involving  information  flow  enabling  enhanced  operational  effectiveness 
via  cooperative  control  in  autonomous  systems.  These  dynamical  network  systems  cover  a 
very  broad  spectrum  of  applications  including  cooperative  control  of  unmanned  air  vehicles 
(UAV’s)  and  autonomous  underwater  vehicles  (AUV’s)  for  combat,  surveillance,  and  recon¬ 
naissance;  distributed  reconfigurable  sensor  networks  for  managing  power  levels  of  wireless 
networks;  air  and  ground  transportation  systems  for  air  traffic  control  and  payload  transport 
and  traffic  management;  swarms  of  air  and  space  vehicle  formations  for  command  and  con¬ 
trol  between  heterogeneous  air  and  space  vehicles;  and  congestion  control  in  communication 
networks  for  routing  the  flow  of  information  through  a  network. 

To  enable  the  applications  for  these  multiagent  systems,  cooperative  control  tasks  such 
as  formation  control,  rendezvous,  flocking,  cyclic  pursuit,  cohesion,  separation,  alignment, 
and  consensus  need  to  be  developed.  To  realize  these  tasks,  individual  agents  need  to  share 
information  of  the  system  objectives  as  well  as  the  dynamical  network.  In  particular,  in  many 
applications  involving  multiagent  systems,  groups  of  agents  are  required  to  agree  on  certain 
quantities  of  interest.  Information  consensus  over  dynamic  information-exchange  topologies 
guarantees  agreement  between  agents  for  a  given  coordination  task.  Distributed  consensus 
algorithms  involve  neighbor- to-neighbor  interaction  between  agents  wherein  agents  update 
their  information  state  based  on  the  information  states  of  the  neighboring  agents.  A  unique 
feature  of  the  closed-loop  dynamics  under  any  control  algorithm  that  achieves  consensus  in 
a  dynamical  network  is  the  existence  of  a  continuum  of  equilibria  representing  a  state  of 
consensus.  Under  such  dynamics,  the  limiting  consensus  state  achieved  is  not  determined 
completely  by  the  dynamics,  but  depends  on  the  initial  state  as  well.  As  discussed  in 
Section  2.4,  in  systems  possessing  a  continuum  of  equilibria,  semistability,  and  not  asymptotic 
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stability  is  the  relevant  notion  of  stability. 

Even  though  many  consensus  protocol  algorithms  have  been  developed  over  the  last  sev¬ 
eral  years  in  the  literature,  robustness  properties  of  these  algorithms  have  been  ignored. 
Robustness  here  refers  to  sensitivity  of  the  control  algorithm  achieving  semistability  and 
consensus  in  the  face  of  communication  uncertainty  and  communication  dropouts.  In  this 
research  [30],  we  build  on  the  results  of  [17,19]  to  develop  robust  control  algorithms  for  net¬ 
work  consensus  protocols  with  uncertain  communication  topologies  of  a  specified  structure. 
In  particular,  we  construct  homogeneous  control  protocol  functions  that  scale  in  a  consistent 
fashion  with  respect  to  a  scaling  operation  on  an  underlying  space  with  the  additional  prop¬ 
erty  that  the  protocol  functions  can  be  written  as  a  sum  of  functions,  each  homogeneous 
with  respect  to  a  fixed  scaling  operation,  that  retain  system  semistability  and  consensus. 

2.7,  Finite-Time  Stabilization  of  Large-Scale  Nonlinear  Dynami¬ 
cal  Systems 

The  notions  of  asymptotic  and  exponential  stability  in  dynamical  systems  theory  imply 
convergence  of  the  system  trajectories  to  an  equilibrium  state  over  the  infinite  horizon.  In 
many  applications,  however,  it  is  desirable  that  a  dynamical  system  possesses  the  property 
that  trajectories  that  converge  to  a  Lyapunov  stable  equilibrium  state  must  do  so  in  finite 
time  rather  than  merely  asymptotically.  Most  of  the  existing  control  techniques  in  the 
literature  ensure  that  the  closed-loop  system  dynamics  of  a  controlled  system  are  Lipschitz 
continuous,  which  implies  uniqueness  of  system  solutions  in  forward  and  backward  times. 
Hence,  convergence  to  an  equilibrium  state  is  achieved  over  an  infinite  time  interval.  In 
order  to  achieve  convergence  in  finite  time,  the  closed-loop  system  dynamics  need  to  be 
non-Lipschitzian  giving  rise  to  11011-uniqueness  of  solutions  in  backward  time.  Uniqueness  of 
solutions  in  forward  time,  however,  can  be  preserved  in  the  case  of  finite-time  convergence. 

In  this  research  [20],  we  develop  a  general  framework  for  finite-time  stability  analysis  of 
nonlinear  dynamical  systems  using  vector  Lyapunov  functions.  Specifically,  we  construct  a 
vector  comparison  system  that  is  finite-time  stable  and,  using  the  vector  comparison  prin¬ 
ciple  [2],  relate  this  finite-time  stability  property  to  the  stability  properties  of  the  nonlinear 
dynamical  system.  Furthermore,  we  design  universal  finite-time  stabilizing  decentralized 
controllers  for  large-scale  dynamical  systems  based  on  the  newly  proposed  notion  of  a  con¬ 
trol  vector  Lyapunov  function  [2].  I11  addition,  we  present  necessary  and  sufficient  conditions 
for  continuity  of  such  controllers.  Moreover,  we  specialize  these  results  to  the  case  of  a  scalar 
Lyapunov  function  to  obtain  universal  finite-time  stabilizers  for  nonlinear  systems  that  are 
affine  in  the  control. 


11 


2.8.  Complexity,  Robustness,  Self-Organization,  Swarms,  and  Sys¬ 
tem  Thermodynamics 

Due  to  technological  advances  in  sensing,  actuation,  communication,  and  computation 
over  the  last  several  years,  a  considerable  research  effort  has  been  devoted  to  the  control  of 
networks  and  control  over  networks.  Network  systems  involve  distributed  decision-making 
for  coordination  of  dynamic  agents  involving  information  flow  enabling  enhanced  operational 
effectiveness  via  cooperative  control  in  autonomous  systems.  These  dynamical  network  sys¬ 
tems  cover  a  very  broad  spectrum  of  applications  including  cooperative  control  of  unmanned 
air  vehicles  (UAV’s)  and  autonomous  underwater  vehicles  (AUV’s)  for  combat,  surveillance, 
and  reconnaissance;  distributed  reconfigurable  sensor  networks  for  managing  power  levels  of 
wireless  networks*  air  and  ground  transportation  systems  for  air  traffic  control  and  payload 
transport  and  traffic  management;  swarms  of  air  and  space  vehicle  formations  for  command 
and  control  between  heterogeneous  air  and  space  vehicles;  and  congestion  control  in  com¬ 
munication  networks  for  routing  the  flow  of  information  through  a  network. 

To  enable  the  autonomous  operation  for  these  multiagent  systems,  the  development  of 
functional  algorithms  for  agent  coordination  and  control  is  needed.  In  particular,  control 
algorithms  need  to  address  agent  interactions,  cooperative  and  non-cooperative  control,  task 
assignments,  and  resource  allocations.  To  realize  these  tasks,  appropriate  sensory  and  cogni¬ 
tive  capabilities  such  as  adaptation,  learning,  decision-making,  and  agreement  (or  consensus) 
on  the  agent  and  multiagent  levels  are  required.  The  common  approach  for  addressing  the 
autonomous  operation  of  multiagent  systems  is  using  distributed  control  algorithms  involv¬ 
ing  neighbor-to-neighbor  interaction  between  agents  wherein  agents  update  their  information 
state  based  on  the  information  states  of  the  neighboring  agents.  Since  most  multiagent  net¬ 
work  systems  are  highly  interconnected  and  mutually  interdependent,  both  physically  and 
through  a  multitude  of  information  and  communication  networks,  these  systems  are  char¬ 
acterized  by  high-dimensional,  large-scale  interconnected  dynamical  systems.  To  develop 
distributed  methods  for  control  and  coordination  of  autonomous  multiagent  systems,  many 
researchers  have  looked  to  autonomous  swann  systems  appearing  in  nature  for  inspiration. 

Biology  has  shown  that  many  species  of  animals  such  as  insect  swarms,  ungulate  flocks, 
fish  schools,  ant  colonies,  and  bacterial  colonies  self- organize  in  nature.  These  biological 
aggregations  give  rise  to  remarkably  complex  global  behaviors  from  simple  local  interactions 
between  large  numbers  of  relatively  unintelligent  agents  without  the  need  for  centralized 
control.  The  spontaneous  development  (i.e.,  self-organization)  of  these  autonomous  biological 
systems  and  their  spatio-temporal  evolution  to  more  complex  states  often  appears  without 
any  external  system  interaction.  In  other  words,  structure  morphing  into  coherent  groups 
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is  internal  to  the  system  and  results  from  local  interactions  among  subsystem  components 
that  are  independent  of  the  physical  nature  of  the  individual  components.  These  local 
interactions  often  comprise  a  simple  set  of  rules  that  lead  to  remarkably  complex  global 
behaviors.  Complexity  here  refers  to  the  quality  of  a  system  wherein  interacting  subsystems 
self-organize  to  form  hierarchical  evolving  structures  exhibiting  emergent  system  properties. 
Hence,  a  complex  dynamical  system  is  a  system  that  is  greater  than  the  sum  of  its  subsystems 
or  parts.  In  addition,  the  spatially  distributed  sensing  and  actuation  control  architecture 
prevalent  in  such  systems  is  inherently  robust  to  individual  subsystem  (or  agent)  failures 
and  unplanned  behavior  at  the  individual  subsystem  (or  agent)  level. 

The  connection  between  the  local  subsystem  interactions  and  the  globally  complex  system 
behavior  is  often  elusive.  Complex  dynamical  systems  involving  self-organizing  components 
forming  spatio-teinporally  evolving  structures  that  exhibit  a  hierarchy  of  emergent  system 
properties  are  not  limited  to  biological  aggregation  systems.  Such  systems  include,  for  ex¬ 
ample,  nervous  systems,  immune  systems,  ecological  systems,  quantum  particle  systems, 
chemical  reaction  systems,  economic  systems,  cellular  systems,  and  galaxies,  to  cite  but  a 
few  examples.  These  systems  are  known  as  dissipative  systems  [24]  and  consume  energy  and 
matter  while  maintaining  their  stable  structure  by  dissipating  entropy  to  the  environment. 
For  example,  as  in  biology,  in  the  physical  universe  billions  of  stars  and  galaxies  interact  to 
form  self-organizing  dissipative  nonequilibrium  structures.  The  fundamental  common  phe¬ 
nomenon  among  these  systems  are  that  they  evolve  in  accordance  to  the  laws  of  (nonequilib- 
riuin)  thermodynamics  which  are  among  the  most  firmly  established  laws  of  nature.  System 
thermodynamics,  in  the  sense  of  [10],  involves  open  interconnected  dynamical  systems  that 
exchange  matter  and  energy  with  their  environment  in  accordance  with  the  first  law  (con¬ 
servation  of  energy)  and  the  second  law  (nonconservation  of  entropy)  of  thermodynamics. 
Self-organization  can  spontaneously  occur  in  such  systems  by  invoking  the  two  fundamental 
axioms  of  the  science  of  heat.  Namely,  i)  if  the  energies  in  the  connected  subsystems  of 
an  interconnected  system  are  equal,  then  energy  exchange  between  these  subsystems  is  not 
possible,  and  ii)  energy  flows  from  more  energetic  subsystems  to  less  energetic  subsystems. 
These  axioms  establish  the  existence  of  a  system  entropy  function  as  well  as  equipartition 
of  energy  [10]  in  system  thermodynamics  and  infoimation  consensus  [17,  19]  in  coopera¬ 
tive  networks;  an  emergent  behavior  in  thermodynamic  systems  as  well  as  swarm  systems. 
Hence,  in  complex  interconnected  dynamical  systems,  self-organization  is  not  a  property  of 
the  system’s  parts  but  rather  emerges  as  a  result  of  the  nonlinear  subsystem  interactions. 

In  light  of  the  above  discussion,  engineering  swarm  systems  necessitates  the  development 
of  relatively  simple  autonomous  agents  that  are  inherently  distributed,  self-organized,  and 
truly  scalable.  Scalability  follows  from  the  fact  that  such  systems  do  not  involve  centralized 
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control  and  communication  architectures.  In  addition,  engineered  swarming  systems  should 
be  inherently  robust  to  individual  agent  failures,  unplanned  task  assignment  changes,  and 
environmental  changes.  Mathematical  models  for  large-scale  swarms  can  involve  Lagrangian 
and  Eulerian  models.  In  a  Lagrangian  model,  each  agent  is  modeled  as  a  particle  governed 
by  a  difference  or  differential  equation,  whereas  an  Eulerian  model  describes  the  local  energy 
01  information  flux  for  a  distribution  of  swarms  with  an  advection-diffusion  (conservation) 
equation.  The  two  formulations  can  be  connected  by  a  Fokker-Plank  approximation  relating 
jump  distance  distributions  of  individual  agents  to  terms  in  the  advection-diffusion  equation. 

In  many  applications  involving  multiagent  systems,  groups  of  agents  are  required  to 
agree  on  certain  quantities  of  interest.  In  particular,  it  is  important  to  develop  informa¬ 
tion  consensus  protocols  for  networks  of  dynamic  agents  wherein  a  unique  feature  of  the 
closed-loop  dynamics  under  any  control  algorithm  that  achieves  consensus  is  the  existence 
of  a  continuum  of  equilibria  representing  a  state  of  equipartitioning  or  consensus.  Under 
such  dynamics,  the  limiting  consensus  state  achieved  is  not  determined  completely  by  the 
dynamics,  but  depends  on  the  initial  system  state  as  well.  For  such  systems  possessing  a 
continuum  of  equilibria,  semistability  [9],  and  not  asymptotic  stability,  is  the  relevant  no¬ 
tion  of  stability.  Semistability  is  the  property  whereby  every  trajectory  that  starts  in  a 
neighborhood  of  a  Lyapunov  stable  equilibrium  converges  to  a  (possibly  different  )  Lyapunov 
stable  equilibrium.  From  a  practical  viewpoint,  it  is  not  sufficient  to  only  guarantee  that  a 
swarm  converges  to  a  state  of  consensus  since  steady  state  convergence  is  not  sufficient  to 
guarantee  that  small  perturbations  from  the  limiting  state  will  lead  to  only  small  transient 
excursions  from  a  state  of  consensus.  It  is  also  necessary  to  guarantee  that  the  equilibrium 
states  representing  consensus  are  Lyapunov  stable,  and  consequently,  semistable. 

In  this  research  [24],  we  develop  distributed  boundary  control  algorithms  for  addressing 
the  consensus  problem  for  an  Eulerian  swarm  model.  The  proposed  distributed  boundary 
controller  architectures  are  predicated  on  the  recently  developed  notion  of  system  thermo¬ 
dynamics  [10]  resulting  in  controller  architectures  involving  the  exchange  of  information 
between  uniformly  distributed  swarms  over  an  n-dimensional  (not  necessarily  Euclidian) 
space  that  guarantee  that  the  closed-loop  system  is  consistent  with  basic  thermodynamic 
principles.  For  our  thermodynamically  consistent  model  we  further  establish  the  existence 
of  a  unique  continuously  differentiable  entropy  functional  for  all  equilibrium  and  nonequi- 
libriuni  states  of  our  system.  Information  consensus  and  semistability  are  shown  using  the 
well-known  Sobolev  embedding  theorems  and  the  notion  of  generalized  (or  weak)  solutions. 
Finally,  since  the  closed-loop  system  is  guaranteed  to  satisfy  basic  thermodynamic  prin¬ 
ciples,  robustness  to  individual  agent  failures  and  unplanned  individual  agent  behavior  is 
automatically  guaranteed. 
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2.9.  Semistability,  Differential  Inclusions,  and  Consensus  Proto¬ 
cols  for  Dynamical  Networks  with  Switching  Topology 


Modern  complex  dynamical  systems  are  highly  interconnected  and  mutually  interdepen¬ 
dent,  both  physically  and  through  a  multitude  of  information  and  communication  networks. 
Distributed  decision-making  for  coordination  of  networks  of  dynamic  agents  involving  infor¬ 
mation  flow  can  be  naturally  captured  by  graph-theoretic  notions.  As  noted  in  Section  2.8, 
these  dynamical  network  systems  cover  a  very  broad  spectrum  of  applications  including  coop¬ 
erative  control  of  unmanned  air  vehicles  (UAV’s),  autonomous  underwater  vehicles  (AUV’s). 
distributed  sensor  networks,  air  and  ground  transportation  systems,  swarms  of  air  and  space 
vehicle  formations,  and  congestion  control  in  communication  networks,  to  cite  but  a  few 
examples.  Hence,  it  is  not  surprising  that  a  considerable  research  effort  has  been  devoted  to 
control  of  networks  and  control  over  networks  in  recent  years. 

Since  communication  links  among  multiagent  systems  are  often  unreliable  due  to  mul¬ 
tipath  effects  and  exogenous  disturbances,  the  information  exchange  topologies  in  network 
systems  are  often  dynamic.  In  particular,  link  failures  or  creations  in  network  multiagent 
systems  result  in  switchings  of  the  communication  topology.  This  is  the  case,  for  example, 
if  information  between  agents  is  exchanged  by  means  of  line-of-sight  sensors  that  experience 
periodic  communication  dropouts  due  to  agent  motion.  Variation  in  network  topology  in¬ 
troduces  control  input  discontinuities,  which  in  turn  give  rise  to  discontinuous  dynamical 
systems.  In  addition,  the  communication  topology  may  be  time-varying.  In  this  case,  the 
vector  field  defining  the  dynamical  system  is  a  discontinuous  function  of  the  state  and  time, 
and  hence,  system  stability  can  be  analyzed  using  nonsmooth  Lyapunov  theory  involving 
concepts  such  as  weak  and  strong  stability  notions,  differential  inclusions,  and  generalized 
gradients  of  locally  Lipschitz  functions  and  proximal  subdifferentials  of  lower  sernicontinuous 
functions. 

In  many  applications  involving  multiagent  systems,  groups  of  agents  are  required  to  agree 
on  certain  quantities  of  interest.  In  particular,  it  is  important  to  develop  information  con¬ 
sensus  protocols  for  networks  of  dynamic  agents  wherein  a  unique  feature  of  the  closed-loop 
dynamics  under  any  control  algorithm  that  achieves  consensus  is  the  existence  of  a  continuum 
of  equilibria  representing  a  state  of  equipartitioning  or  consensus .  Under  such  dynamics,  the 
limiting  consensus  state  achieved  is  not  determined  completely  by  the  dynamics,  but  depends 
on  the  initial  system  state  as  well.  For  such  systems  possessing  a  continuum  of  equilibria, 
sernistability  [9],  and  not  asymptotic  stability,  is  the  relevant  notion  of  stability. 

To  address  agreement  problems  in  switching  networks  with  time-dependent  and  state- 
dependent  topologies,  in  this  research  [36,  74]  we  extend  the  theory  of  semistability  to  dis- 
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continuous  time-invariant  and  time-varying  dynamical  systems.  In  particular,  we  develop 
necessary  and  sufficient  conditions  to  guarantee  weak  and  strong  invariance  of  Fillipov  solu¬ 
tions  under  the  assumption  that  the  discontinuous  system  vector  field  is  uniformly  bounded. 
Moreover,  we  present  Lvapunov-based  tests  for  strong  semistability,  weak  semistability,  as 
well  as  uniform  semistability  for  autonomous  and  nonautonomous  differential  inclusions. 

2.10.  H2  Optimal  Semistable  Control  for  Linear  Dynamical  Sys¬ 
tems 

Dynamical  network  systems  cover  a  very  broad  spectrum  of  applications  including  coop¬ 
erative  control  of  unmanned  air  vehicles,  autonomous  underwater  vehicles,  distributed  sensor 
networks,  air  and  ground  transportation  systems,  swarms  of  air  and  space  vehicle  formations, 
and  congestion  control  in  communication  networks,  to  cite  but  a  few  examples.  A  unique 
feature  of  the  closed-loop  dynamics  under  any  control  algorithm  in  dynamical  networks  is 
the  existence  of  a  continuum  of  equilibria  representing  a  desired  state  of  convergence.  Un¬ 
der  such  dynamics,  the  desired  limiting  state  is  not  determined  completely  by  the  system 
dynamics,  but  depends  on  the  initial  system  state  as  well  [17,19]. 

The  dependence  of  the  limiting  state  on  the  initial  state  is  not  limited  to  dynamical 
network  systems,  it  is  also  seen  in  the  dynamics  of  coinpartinental  systems  which  arise  in 
chemical  kinetics,  and  biomedical,  environmental,  economic,  power,  and  thermodynamic 
systems.  In  all  such  systems  possessing  a  continuum  of  equilibria,  semistability,  and  not 
asymptotic  stability,  is  the  relevant  notion  of  stability. 

In  this  research  [28,35],  we  use  linear  matrix  inequalities  (LMIs)  to  develop  0,2  opti¬ 
mal  semistable  controllers  for  linear  dynamical  systems.  Linear  matrix  inequalities  provide 
a  powerful  design  framework  for  linear  control  problems.  Since  LMIs  lead  to  convex  or 
quasiconvex  optimization  problems,  they  can  be  solved  very  efficiently  using  interior-point 
algorithms.  Unlike  the  standard  H2  optimal  control  problem,  a  complicating  feature  of  the 
0,2  optimal  semistable  stabilization  problem  is  that  the  closed-loop  Lyapunov  equation  guar¬ 
anteeing  semistability  can  admit  multiple  solutions.  An  interesting  feature  of  the  proposed 
approach,  however,  is  that  a  least  squares  solution  over  all  possible  semistabilizing  solutions 
corresponds  to  the  H2  optimal  solution.  It  is  shown  that  this  least  squares  solution  can  be 
characterized  by  a  linear  matrix  inequality  minimization  problem. 
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2.11.  Finite-Time  Stabilization  for  Nonlinear  Impulsive  Dynami¬ 
cal  Systems 

The  mathematical  descriptions  of  many  hybrid  dynamical  systems  can  be  characterized 
by  impulsive  differential  equations  [1].  Impulsive  dynamical  systems  can  be  viewed  as  a  sub¬ 
class  of  hybrid  systems  and  consist  of  three  elements-  namely,  a  eontinuous-time  differential 
equation,  which  governs  the  motion  of  the  dynamical  system  between  impulsive  or  resetting 
events;  a  difference  equation,  which  governs  the  way  the  system  states  are  instantaneously 
changed  when  a  resetting  event  occurs;  and  a  criterion  for  determining  when  the  states  of 
the  system  are  to  be  reset.  Since  impulsive  systems  ean  involve  impulses  at  variable  times, 
they  are  in  general  time-varying  systems,  wherein  the  resetting  events  are  both  a  function 
of  time  and  the  system's  state.  In  the  case  where  the  resetting  events  are  defined  by  a 
prescribed  sequence  of  times  which  are  independent  of  the  system  state,  the  equations  are 
known  as  time- dependent  differential  equations  [1].  Alternatively,  in  the  case  where  the  re¬ 
setting  events  are  defined  by  a  manifold  in  the  state  space  that  is  independent  of  time,  the 
equations  are  autonomous  and  are  known  as  state- dependent  differential  equations  [1]. 

Finite-time  stability  implies  Lyapunov  stability  and  convergence  of  system  trajectories 
to  an  equilibrium  state  in  finite-time,  and  hence,  is  a  stronger  notion  than  asymptotie  sta¬ 
bility.  For  continuous-time  dynamical  systems,  finite-time  stability  implies  non-Lipschitzian 
dynamics  [9]  giving  rise  to  non-uniqueness  of  solutions  in  reverse  time.  Uniqueness  of  so¬ 
lutions  in  forward  time,  however,  ean  be  preserved  in  the  ease  of  finite-time  convergence. 
Finite-time  convergence  to  a  Lyapunov  stable  equilibrium  for  continuous-time  systems,  that 
is,  finite-time  stability,  was  rigorously  studied  in  [9]  using  Holder  continuous  Lyapunov  func¬ 
tions. 

Finite-time  stability  of  impulsive  dynainieal  systems,  however,  has  not  been  studied  in 
the  literature.  For  impulsive  dynamical  systems,  it  may  be  possible  to  reset  the  system  states 
to  an  equilibrium  state,  in  which  case  finite-time  convergence  of  the  system  trajectories  can 
be  achieved  without  the  requirement  for  non-Lipschitzian  dynamics.  In  addition,  due  to 
system  resettings,  impulsive  dynamical  systems  may  exhibit  non-uniqueness  of  solutions  in 
reverse  time  even  when  the  continuous-time  dynamics  are  Lipschitz  continuous. 

In  this  research  [15],  we  develop  sufficient  conditions  for  finite-time  stability  of  nonlinear 
impulsive  dynamical  systems.  Furthermore,  we  present  stability  results  using  vector  Lya¬ 
punov  functions  wherein  finite-time  stability  of  the  impulsive  system  is  shown  via  finite-time 
stability  of  a  hybrid  comparison  system.  We  use  these  results  to  further  construct  hybrid 
finite-time  stabilizing  controllers  for  impulsive  dynamical  systems.  In  addition,  we  construct 
decentralized  finite-time  stabilizers  for  large-scale  impulsive  dynamical  systems. 
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2.12.  Neural  Network  Hybrid  Adaptive  Control  for  Nonlinear  Un¬ 
certain  Impulsive  Dynamical  Systems 

Modern  complex  engineering  systems  involve  multiple  modes  of  operation  placing  strin¬ 
gent  demands  on  controller  design  and  implementation  of  increasing  complexity.  Such  sys¬ 
tems  typically  possess  a  multiechelon  hierarchical  hybrid  control  architecture  characterized 
by  continuous-time  dynamics  at  the  lower  levels  of  the  hierarchy  and  discrete-time  dynamics 
at  the  higher  levels  of  the  hierarchy  1].  The  lower-level  units  directly  interact  with  the 
dynamical  system  to  be  controlled  while  the  higher-level  units  receive  information  from  the 
lower-level  units  as  inputs  and  provide  (possibly  discrete)  output  commands  which  serve  to 
coordinate  and  reconcile  the  (sometimes  competing)  actions  of  the  lower-level  units.  The  hi¬ 
erarchical  controller  organization  reduces  processor  cost  and  controller  complexity  by  break¬ 
ing  up  the  processing  task  into  relatively  small  pieces  and  decomposing  the  fast  and  slow 
control  functions.  Typically,  the  higher-level  units  perforin  logical  checks  that  determine  sys¬ 
tem  mode  operation,  while  the  lower-level  units  execute  continuous-variable  commands  for 
a  given  system  mode  of  operation.  The  mathematical  description  of  many  of  these  systems 
can  be  characterized  by  impulsive  differential  equations  [1]. 

The  purpose  of  feedback  control  is  to  achieve  desirable  system  performance  in  the  face 
of  system  uncertainty.  To  this  end,  adaptive  control  along  with  robust  control  theory  have 
been  developed  to  address  the  problem  of  system  uncertainty  in  control-system  design.  In 
contrast  to  fixed-gain  robust  controllers,  which  maintain  specified  constants  within  the  feed¬ 
back  control  law  to  sustain  robust  performance,  adaptive  controllers  directly  or  indirectly 
adjust  feedback  gains  to  maintain  closed-loop  stability  and  improve  performance  in  the  face 
of  system  uncertainties.  Specifically,  indirect  adaptive  controllers  utilize  parameter  update 
laws  to  identify  unknown  system  parameters  and  adjust  feedback  gains  to  account  for  system 
variation,  while  direct  adaptive  controllers  directly  adjust  the  controller  gains  in  response 
to  plant  variations.  The  inherent  nonlinearities  and  system  uncertainties  in  hierarchical  hy¬ 
brid  control  systems  and  the  increasingly  stringent  performance  requirements  required  for 
controlling  such  modern  complex  embedded  systems  necessitates  the  development  of  hybrid 
adaptive  nonlinear  control  methodologies. 

Neural  network-based  adaptive  control  algorithms  have  been  extensively  developed  in 
the  literature,  wherein  Lyapunov-like  functions  are  used  to  ensure  that  the  neural  network 
controllers  can  guarantee  ultimate  boundedness  of  the  closed-loop  system  states  rather  than 
closed-loop  asymptotic  stability.  Ultimate  boundness  ensures  that  the  plant  states  converge 
to  a  neighborhood  of  the  origin  [9].  The  reason  why  stability  in  the  sense  of  Lyapunov  is 
not  guaranteed  stems  from  the  fact  that  the  uncertainties  in  the  system  dynamics  cannot  be 
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perfectly  captured  by  neural  networks  using  the  universal  function  approximation  property 
and  the  residual  approximation  error  is  characterized  via  a  norm  hound  over  a  given  com¬ 
pact  set.  Ultimate  boundedness  guarantees,  however,  are  often  conservative  since  standard 
Lyapunov-like  theorems  that  are  typically  used  to  show  ultimate  boundedness  of  the  closed- 
loop  hybrid  system  states  provide  only  sufficient  conditions ,  while  neural  network  controllers 
may  possibly  achieve  plant  state  convergence  to  an  equilibrium  point. 

In  this  research  16],  we  develop  a  neural  hybrid  adaptive  control  framework  for  a  class 
of  nonlinear  uncertain  impulsive  dynamical  systems  which  ensures  state  convergence  to  a 
Lyapunov  stable  equilibrium  as  well  as  boundedness  of  the  neural  network  weighting  gains. 
Specifically,  the  proposed  framework  is  Lyapunov-based  and  guarantees  partial  asymptotic 
stability  of  the  closed-loop  hybrid  system;  that  is,  Lyapunov  stability  of  the  overall  closed- 
loop  states  and  convergence  of  the  plant  state.  The  neuroadaptive  controllers  are  constructed 
without  requiring  explicit  knowledge  of  the  hybrid  system  dynamics  other  than  the  fact  that 
the  plant  dynamics  are  continuously  differentiable  and  that  the  approximation  error  of  the 
unknown  system  nonlinearities  lies  in  a  small  gain-type  norm  bounded  conic  sector  over 
a  compact  set.  Hence,  the  overall  neuroadaptive  control  framework  captures  the  residual 
approximation  error  inherent  in  linear  parameterizations  of  system  uncertainty  via  basis 
functions.  Furthermore,  the  proposed  neuroadaptive  control  architecture  is  modular  in  the 
sense  that  if  a  nominal  linear  design  model  is  available,  then  the  neuroadaptive  controller 
can  be  augmented  to  the  nominal  design  to  account  for  system  nonlinearities  and  system 
uncertainty. 

Finally,  we  emphasize  that  we  do  not  impose  any  linear  growth  condition  on  the  system 
resetting  (discrete)  dynamics.  In  the  literature  on  classical  (non-neural)  adaptive  control 
theory  for  discrete-time  systems,  it  is  typically  assumed  that  the  nonlinear  system  dynamics 
have  the  linear  growth  rate  which  is  necessary  in  proving  Lyapunov  stability  rather  than 
practical  stability  (ultimate  boundedness).  Our  novel  characterization  of  the  system  uncer¬ 
tainties  (i.e.,  the  small  gain- type  bound  on  the  norm  of  the  modeling  error)  allows  us  to  prove 
asymptotic  stability  without  requiring  a  linear  growth  condition  on  the  system  dynamics. 

2.13.  Controller  Synthesis  with  Guaranteed  Closed-loop  Phase 
Constraints 

The  ability  to  address  gain  and  phase  uncertainties  is  essential  for  maximizing  achievable 
performance  in  controlling  uncertain  dynamical  systems.  The  small  gain  theorem  guarantees 
robust  stability  by  requiring  that  the  loop  gain  (including  desired  weighing  functions  for  loop 
shaping)  be  less  than  unity  at  all  frequencies.  The  small  gain  theorem,  however,  does  not 
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make  use  of  phase  information  in  guaranteeing  stability.  To  some  extent,  phase  information 
is  accounted  for  by  means  of  positivity  theory.  In  this  theory,  a  positive  real  plant  and  a 
strictly  positive  real  uncertainty  are  both  assumed  to  have  phase  less  than  90°  so  that  the 
loop  transfer  function  has  less  than  180°  of  phase  shift,  hence  guaranteeing  robust  stability 
in  spite  of  gain  uncertainty.  Other  notable  results  addressing  phase  information  include 
concepts  such  as  principal  phases,  multivariable  phase  margin,  phase  spread,  phase  envelope, 
phase  matching,  phase-sensitive  structured  singular  value,  and  plant  uncertainty  templates. 
With  the  exception  of  positivity  theory  all  of  the  aforementioned  methods  are  restricted 
to  frequency  domain  characterizations  and  are  not  amenable  to  state  space  formulations 
necessary  for  developing  controller  synthesis  methods  with  guaranteed  phase  constraints. 

In  this  research  [22],  we  present  an  analysis  and  synthesis  approach  for  guaranteeing  that 
the  phase  of  a  single-input,  single-output  closed-loop  transfer  function  is  contained  in  the 
interval  [— cv,a]  for  a  given  a  >  0  at  all  frequencies.  Specifically,  we  first  derive  a  sufficient 
condition  involving  a  frequency  domain  inequality  for  guaranteeing  a  given  phase  constraint. 
Next,  we  use  the  Kalman- Yakubovich-Popov  (KYP)  theorem  to  derive  an  equivalent  time 
domain  condition.  In  the  case  where  a  =  we  show  that  frequency  and  time  domain 
sufficient  conditions  specialize  to  the  positivity  theorem.  Furthermore,  using  linear  matrix 
inequalities  (LMIs),  we  develop  a  controller  synthesis  approach  for  guaranteeing  a  phase 
constraint  on  the  closed-loop  transfer  function.  Finally,  we  extend  this  synthesis  approach 
to  address  mixed  gain  and  phase  constraints  on  the  closed-loop  transfer  function. 

2.14.  Adaptive  Control  for  Nonlinear  Uncertain  Systems  with  Ac¬ 
tuator  Amplitude  and  Rate  Saturation  Constraints 

In  light  of  the  increasingly  complex  and  highly  uncertain  nature  of  dynamical  systems  re¬ 
quiring  controls,  it  is  not  surprising  that  reliable  system  models  for  many  high  performance 
engineering  applications  are  unavailable.  In  the  face  of  such  high  levels  of  system  uncer¬ 
tainty,  robust  controllers  may  unnecessarily  sacrifice  system  performance,  whereas  adaptive 
controllers  are  clearly  appropriate  since  they  can  tolerate  far  greater  system  uncertainty 
levels  to  improve  system  performance.  However,  an  implicit  assumption  inherent  in  most 
adaptive  control  frameworks  is  that  the  adaptive  control  law  is  implemented  without  any 
regard  to  actuator  amplitude  and  rate  saturation  constraints.  Of  course,  any  electrome¬ 
chanical  control  actuation  device  is  subject  to  amplitude  and/or  rate  constraints  leading  to 
saturation  nonlinearities  enforcing  limitations  on  control  amplitudes  and  control  rates.  As 
a  consequence,  actuator  nonlinearities  arise  frequently  in  practice  and  can  severely  degrade 
closed-loop  system  performance,  and  in  some  cases  drive  the  system  to  instability.  These 
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effects  are  even  more  pronounced  for  adaptive  controllers  which  continue  to  adapt  when  the 
feedback  loop  has  been  severed  due  to  the  presence  of  actuator  saturation  causing  unstable 
controller  modes  to  drift,  which  in  turn  leads  to  severe  windup  effects. 

Many  practical  applications  involve  nonlinear  dynamical  systems  with  simultaneous  con¬ 
trol  amplitude  and  rate  saturation.  The  presence  of  control  rate  saturation  may  further 
exacerbate  the  problem  of  control  amplitude  saturation.  For  example,  in  advanced  tacti¬ 
cal  fighter  aircraft  with  high  maneuverability  requirements,  pilot  induced  oscillations  can 
cause  actuator  amplitude  and  rate  saturation  in  the  control  surfaces,  leading  to  catastrophic 
failures. 

In  this  research  [26],  we  develop  a  direct  adaptive  control  framework  for  adaptive  tracking 
of  multivariable  nonlinear  uncertain  systems  with  amplitude  and  rate  saturation  constraints. 
In  particular,  we  extend  the  Lyapunov-based  direct  adaptive  control  framework  developed  in 
[11]  to  guarantee  asymptotic  stability  of  the  closed-loop  tracking  system;  that  is,  asymptotic 
stability  with  respect  to  the  closed-loop  system  states  associated  with  the  tracking  error 
dynamics  in  the  face  of  actuator  amplitude  and  rate  saturation  constraints.  Specifically,  a 
reference  (governor  or  supervisor)  dynamical  system  is  constructed  to  address  tracking  and 
regulation  by  deriving  adaptive  update  laws  that  guarantee  that  the  error  system  dynamics 
are  asymptotically  stable,  and  the  adaptive  controller  gains  are  Lyapunov  stable.  In  the 
case  where  the  actuator  amplitude  and  rate  are  limited,  the  adaptive  control  signal  to  the 
reference  system  is  modified  to  effectively  robustify  the  error  dynamics  to  the  saturation 
constraints,  thus  guaranteeing  asymptotic  stability  of  the  error  states. 

2*15.  A  New  Neuroadaptive  Control  Architecture  for  Nonlinear 
Uncertain  Dynamical  Systems 

One  of  the  primary  reasons  for  the  large  interest  in  neural  networks  is  their  capability 
to  approximate  a  large  class  of  continuous  nonlinear  maps  from  the  collective  action  of  very 
simple,  autonomous  processing  units  interconnected  in  simple  ways.  Neural  networks  have 
also  attracted  attention  due  to  their  inherently  parallel  and  highly  redundant  processing 
architecture  that  makes  it  possible  to  develop  parallel  weight  update  laws.  This  parallelism 
makes  it  possible  to  effectively  update  a  neural  network  on  line.  These  properties  make 
neural  networks  a  viable  paradigm  for  adaptive  system  identification  and  control  of  complex 
highly  uncertain  systems,  and  as  a  consequence  the  use  of  neural  networks  for  identification 
and  control  has  become  an  active  area  of  research. 

The  goal  of  adaptive  and  neuroadaptive  control  is  to  achieve  system  performance  without 
excessive  reliance  on  system  models.  Both  controller  approaches  directly  or  indirectly  adjust 
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feedback  controller  gains  and  improve  system  performance  in  the  face  of  system  uncertainty. 
Specifically,  indirect  adaptive  and  neuroadaptive  controllers  utilize  parameter  update  laws 
to  identify  unknown  system  parameters  and  adjust  feedback  gains  to  account  for  system 
variation,  while  direct  adaptive  and  neuroadaptive  controllers  adjust  the  controller  gains  in 
response  to  system  variations. 

The  fundamental  difference  between  adaptive  control  and  neuroadaptive  control  can  be 
traced  back  to  the  modeling  and  treatment  of  the  system  uncertainties.  In  particular,  adap¬ 
tive  control  is  based  on  constant ,  linearly  parameterized  system  uncertainty  models  of  a 
known  structure  but  unknown  variation.  This  uncertainty  characterization  allows  for  the 
system  nonliiiearities  to  be  parameterized  by  a  finite  linear  combination  of  basis  functions 
within  a  class  of  function  approximators  such  as  rational  functions,  spline  functions,  radial 
basis  functions,  sigmoidal  functions,  and  wavelets.  However,  this  linear  parametrization  of 
basis  functions  cannot,  in  general,  exactly  capture  the  unknown  system  parameters.  In  such 
a  case,  the  uncertainty  is  expressed  in  terms  of  a  neural  network  involving  a  parameterized 
nonlinearity.  Hence,  in  contrast  to  adaptive  control,  neuroadaptive  control  is  based  on  the 
universal  function  approximation  property,  wherein  any  continuous  nonlinear  system  uncer¬ 
tainty  can  be  approximated  arbitrarily  closely  on  a  compact  set  using  a  neural  network  with 
appropriate  weights  .  This  difference  in  the  modeling  and  treatment  of  the  system  uncertain¬ 
ties  results  in  the  ability  of  adaptive  controllers  to  guarantee  asymptotic  closed-loop  system 
stability  versus  ultimate  boundness  as  is  the  case  with  neuroadaptive  controllers  11]. 

To  improve  robustness  and  the  speed  of  adaptation  of  adaptive  and  neuroadaptive  con¬ 
trollers  several  controller  architectures  have  been  proposed  in  the  literature.  These  include 
the  a-  and  e-modification  architectures  used  to  keep  the  system  parameter  estimates  from 
growing  without  bound  in  the  face  of  system  uncertainty  [40].  In  this  research  [40],  a  new 
neuroadaptive  control  architecture  for  nonlinear  uncertain  dynamical  systems  is  developed. 
Specifically,  the  proposed  framework  involves  a  new  and  novel  controller  architecture  involv¬ 
ing  additional  terms,  or  Q-rnodification  terms ,  in  the  update  laws  that  are  constructed  using 
a  moving  window  of  the  integrated  system  uncertainty.  The  Q-modification  terms  can  be 
used  to  identify  the  ideal  neural  network  system  weights  which  can  be  used  in  the  adaptive 
law.  In  addition,  these  terms  effectively  suppress  system  uncertainty. 

Even  though  the  proposed  approach  is  reminiscent  to  the  composite  adaptive  control 
framework,  the  Q-modification  framework  does  not  involve  filtered  versions  of  the  control 
input  and  system  state  in  the  update  laws.  Rather,  the  update  laws  involve  auxiliary  terms 
predicated  on  an  estimate  of  the  unknown  neural  network  weights  which  in  turn  are  char¬ 
acterized  by  an  auxiliary  equation  involving  the  integrated  error  dynamics  over  a  moving 
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time  interval.  In  this  research  [40],  we  consider  vector  uncertainty  structures  with  both  lin¬ 
ear  and  nonlinear  parameterizations.  In  addition,  state  and  output  feedback  controllers  are 
developed.  Finally,  to  illustrate  the  efficacy  of  the  proposed  approach  we  apply  our  results 
to  an  aircraft  model  with  wing  rock  dynamics  as  well  as  a  spaeeeraft  model  involving  an 
unknown  moment  of  inertia  matrix  and  compare  our  results  with  standard  neuroadaptive 
control  methods. 

2.16.  H2  Suboptimal  Estimation  and  Control  for  Nonnegative  Dy¬ 
namical  Systems 

Nonnegative  dynamical  systems  involve  dynamie  states  whose  values  are  nonnegative. 
A  subclass  of  nonnegative  dynamical  systems  are  compart  mental  systems.  Cornpartmental 
systems  involve  dynamical  models  that  are  characterized  by  conservation  laws  (e.g.,  mass, 
energy,  fluid,  ete.)  capturing  the  exchange  of  material  between  eoupled  macroseopic  sub¬ 
systems  known  as  compartments.  These  models  are  widespread  in  biological,  physiological, 
and  ecological  sciences  as  well  as  engineering  systems  such  as  queuing,  large-seale,  telecom¬ 
munications,  transportation,  power,  and  network  systems,  to  cite  but  a  few  examples.  Since 
nonnegative  and  eoinpartmental  systems  have  specialized  structures,  special  control  law 
strategies  need  to  be  developed  that  guarantee  that  the  trajectories  of  the  closed-loop  plant 
system  states  remain  in  the  nonnegative  orthant  of  the  state  spaee  for  nonnegative  initial 
conditions.  In  addition,  for  certain  applications  of  nonnegative  systems,  such  as  active  con¬ 
trol  for  clinical  pharmacology,  we  require  the  control  (source)  inputs  to  be  nonnegative. 

Even  though  nonnegative  systems  are  often  encountered  in  numerous  application  ar¬ 
eas.  nonnegative  orthant  stabilizability  and  holdability  has  received  little  attention  in  the 
literature.  I11  this  research  [27],  we  use  linear  matrix  inequalities  (LMIs)  to  develop  H? 
(sub)optirnal  estimators  and  controllers  for  nonnegative  dynamical  systems.  Linear  matrix 
inequalities  provide  a  powerful  design  framework  for  linear  control  problems.  Since  LMIs 
lead  to  convex  or  quasiconvex  optimization  problems,  they  ean  be  solved  very  efficiently 
using  interior-point  algorithms.  An  intersting  feature  of  nonnegative  orthant  stabilizability 
is  that  it  can  be  formulated  as  a  solution  to  an  LMI  problem.  However,  H2  optimal  non- 
negative  orthant  stabilizability  cannot,  in  general,  be  formulated  as  an  LMI  problem.  I11 
this  research  27],  we  formulate  a  series  of  generalized  eigenvalue  problems  subject  to  a  set 
of  LMI  constraints  for  designing  Tiz  suboptimal  estimators,  static  controllers,  and  dynamic 
controllers  for  nonnegative  dynamical  systems. 
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2.17.  Adaptive  Disturbance  Rejection  Control  for  Compartmental 
Systems 

As  discussed  in  Section  2.16,  nonnegative  and  compartmental  systems  are  essential  in 
capturing  the  behavior  of  a  wide  range  of  dynamical  systems  involving  dynamic  states 
whose  values  are  nonnegative.  These  systems  are  derived  from  mass  and  energy  balance 
considerations  and  are  comprised  of  homogeneous  interconnected  microscopic  subsystems  or 
compartments  which  exchange  variable  quantities  of  material  via  intercoinpart mental  How 
laws.  Since  biological  and  physiological  systems  have  numerous  input,  state,  and  output 
properties  related  to  conservation,  dissipation,  and  transport  of  mass  and  energy,  nonnega- 
tive  and  compartmental  systems  are  remarkably  effective  in  describing  the  phenomenological 
behavior  of  these  dynamical  systems.  The  range  of  applications  of  nonnegative  and  com- 
partmental  systems  is  not  limited  to  biological  and  medical  systems.  Their  usage  includes 
demographic,  epidemic,  ecological,  economic,  telecommunications,  transportation,  power, 
and  large-scale  systems. 

In  a  recent  series  by  the  Principal  Investigator  (see  the  references  in  [29])  a  direct  adap¬ 
tive  control  framework  for  linear  and  nonlinear  nonnegative  and  compartmental  systems  was 
developed.  This  framework  is  Lyapunov-based  and  guarantees  partial  asymptotic  set-point 
regulation,  that  is,  asymptotic  set  point  stability  with  respect  to  the  closed-loop  system 
states  associated  with  the  plant.  In  addition,  the  adaptive  controllers  guarantee  that  the 
physical  system  states  remain  in  the  nonnegative  orthant  of  the  state  space.  In  this  re¬ 
search  l25],  we  develop  a  direct  adaptive  control  framework  for  adaptive  stabilization  and 
disturbance  rejection  for  compart  mental  dynamical  systems  with  exogenous  system  distur¬ 
bances.  The  main  challenge  here  is  to  construct  nonlinear  adaptive  disturbance  rejection 
controllers  without  requiring  knowledge  of  the  system  dynamics  or  the  system  disturbances 
while  guaranteeing  that  the  physical  system  states  remain  in  the  nonnegative  orthant  of  the 
state  space. 

While  such  an  adaptive  control  framework  can  have  wide  applicability  in  areas  such  as 
economics,  telecommunications,  and  power  systems,  its  use  in  the  specific  field  of  anesthetic 
pharmacology  is  particularly  noteworthy.  Specifically,  during  stress  (such  as  hemorrhage) 
in  an  acute  care  environment,  such  as  the  operating  room,  perfusion  pressure  falls  and  hy¬ 
pertonic  saline  solutions  are  typically  intravenously  administered  to  regulate  hemodynamic 
effects  and  avoid  hemorrhagic  shock.  This  exogenous  disturbance  drives  the  system  pharma¬ 
cokinetics  and  pharmacodynamics  and  can  be  captured  as  a  system  disturbance.  In  addition, 
exogenous  system  disturbances  can  be  used  to  capture  unmodeled  physiological  and  pharma¬ 
cological  system  dynamics.  Although  the  proposed  framework  develops  adaptive  controllers 
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for  general  cornpartinental  systems  with  exogenous  disturbances,  the  specific  focus  of  the 
research  is  on  pharmacokinetic  models  with  hemorrhage  and  hernodilution  effects. 

2.18.  Neuroadaptive  Output  Feedback  Control  for  Automated  An¬ 
esthesia  with  Noisy  EEG  Measurements 

The  dosing  of  most  drugs  is  a  process  of  empirical  administration  of  a  low  dose  with 
observation  of  the  biological  effect  and  subsequent  adjustment  of  the  dose  in  the  hopes  of 
achieving  the  desired  effect.  This  is  true  of  anesthetic  drugs,  just  as  it  is  of  chronically 
administered  medications  (for  example,  anti- hypertensive  agents).  In  the  acute  environment 
of  the  operating  room  and  intensive  care  unit  (ICU),  this  can  result  in  inefficient  ,  and  possibly 
even  dangerous,  titration  of  drug  to  the  desired  effect.  There  has  been  a  long  interest  in  use  of 
the  electroencephalograph  (EEG)  as  an  objective,  quantitative  measure  of  consciousness  that 
could  be  used  as  a  performance  variable  for  closed-loop  control  of  anesthesia  [29].  Processed 
electroencephalogram  algorithms  have  been  extensively  investigated  as  monitors  of  the  level 
of  consciousness  in  patients  requiring  surgical  anesthesia  [29].  However,  the  EEG  is  a  complex 
of  multiple  time  series  and  in  earlier  work  it  was  difficult  to  identify  one  single  aspect  of  the 
EEG  signal  that  correlated  with  the  clinical  signs  of  anesthesia. 

Subsequent  to  this  early  research  there  has  been  substantial  progress  in  the  development 
of  processed  EEG  monitors  that  analyze  the  raw  data  to  extract  a  single  measure  of  the 
depth  of  anesthesia.  The  best  known  of  these  monitors  is  the  bispectral  or  BIS  monitor, 
which  calculates  a  single  composite  EEG  measure  that  is  well  correlated  with  the  depth  of 
anesthesia  [29].  The  BIS  signal  ranges  from  0  (no  cerebral  electrical  activity)  to  100  (the 
normal  awake  state).  Available  evidence  indicates  that  a  BIS  signal  less  than  55  is  associated 
with  lack  of  consciousness.  While  BIS  monitoring  lias  proven  useful  in  the  operating  room 
environment,  there  have  been  inconsistencies  reported  and  attempts  to  extend  BIS  monitor¬ 
ing  for  the  evaluation  of  sedation  outside  of  the  operating  room  have  been  unsuccessful  [29]. 
One  of  the  key  reasons  for  this  is  due  to  the  fact  that  the  signal- averaging  algorithm  within 
the  BIS  monitor  ignores  signal  noise,  and  when  there  is  excessive  noise,  the  BIS  monitor 
does  not  generate  a  signal. 

It  is  widely  appreciated  that  BIS  monitoring,  or  for  that  matter,  any  EEG  monitoring, 
can  be  fraught  with  error  because  of  the  potential  for  outside  interference  to  produce  an 
unfavorable  signal-to-noise  ratio  yielding  spurious  results.  Nonphysiologic  artifactual  signals 
may  be  generated  from  sources  external  to  the  patient  that  include  lights,  electric  cautery 
devices,  ventilators,  pacemakers,  patient  warming  systems,  and  electrical  noise  related  to 
the  many  different  kinds  of  monitors  normally  found  in  an  operating  room  or  ICU.  Physio- 
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logic  movements  such  as  eye  movements,  myogenic  activity,  perspiration,  and  ventilation  can 
produce  artifactual  increases  in  the  BIS  score.  In  particular,  it  is  apparent  that  electromyo¬ 
graphic  (EMG)  activity  can  spuriously  increase  the  BIS  score  [29],  The  co-administration 
of  neuromuscular  blockade  eliminates  artifacts  from  muscle  movement,  which  can  be  su¬ 
perimposed  on  the  BIS  score;  and  this  undoubtedly  contributes  to  the  widespread  use  and 
value  of  the  BIS  device  during  surgery.  However,  to  extend  this  technology  outside  of  the 
operating  room,  or  for  that  matter,  to  nonparalyzed  patients  in  the  operating  room,  further 
refinements  are  needed.  In  addition,  if  the  BIS  signal  is  to  be  used  to  quantify  levels  of 
consciousness  for  feedback  control  in  general  anesthesia,  then  the  observation  noise  needs  to 
be  accounted  for  in  the  control  system  design  process. 

The  challenge  to  the  use  of  the  BIS  signal  for  closed-loop  control  of  anesthesia  is  that 
the  relationships  between  drug  dose  and  tissue  concentration  (pharmacokinetics)  and  be¬ 
tween  tissue  concentration  and  physiological  effect  (pharmacodynamics)  is  highly  variable 
between  individuals.  In  addition,  observation  noise  in  the  BIS  signal  results  in  feedback 
measurement  signals  with  high  signal-to-noise  ratios  that  need  to  be  accounted  for  in  the 
control  algorithm.  Adaptive  feedback  controllers  seem  particularly  promising  given  this  in- 
terpatient  variability  as  well  as  BIS  signal  variation  due  to  noise.  In  previous  work,  we 
have  used  nonnegative  and  compartmental  dynamical  systems  theory  to  develop  adaptive 
and  neuroadaptive  controllers  for  controlling  the  depth  of  anesthesia  [49].  One  of  our  initial 
efforts  was  the  development  of  a  direct  adaptive  control  framework  for  uncertain  nonlin¬ 
ear  nonnegative  and  compartmental  systems  with  nonnegative  control  inputs  [49].  This 
framework  is  Lyapunov-based  and  guarantees  partial  asymptotic  set-point  regulation,  that 
is,  asymptotic  setpoint  stability  with  respect  to  part  of  the  closed-loop  system  states  asso¬ 
ciated  with  the  physiological  state  variables.  In  addition,  the  adaptive  controllers,  which 
are  constructed  without  requiring  knowledge  of  the  pharmacokinetic  and  pharmacodynamic 
parameters,  provide  a  nonnegative  control  input  for  stabilization  with  respect  to  a  given  set- 
point  in  the  nonnegative  orthant.  Subsequently,  we  also  developed  a  neuroadaptive  output 
feedback  control  framework  for  uncertain  nonlinear  nonnegative  and  compartmental  systems 
with  nonnegative  control  inputs  [7].  This  framework  is  also  Lyapunov-based  and  guarantees 
ultimate  boundedness  of  the  error  signals  corresponding  to  the  physical  system  states  in  the 
face  of  interpatient  pharmacokinetic  and  pharmacodynamic  variability. 

In  a  recent  paper  [49]  we  presented  numerical  and  clinical  results  that  compares  and 
contrasts  our  adaptive  control  algorithm  with  our  neural  network  adaptive  control  algorithm 
for  controlling  the  depth  of  anesthesia  in  the  operating  theater  during  surgery.  Specifically 
eleven  clinical  trials  were  performed  with  our  adaptive  control  algorithm  and  seven  clinical 
trials  were  performed  with  our  neural  network  algorithm  at  the  Northeast  Georgia  Medical 
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Center  in  Gainesville,  Georgia.  The  proposed  automated  anesthesia  controllers  demonstrated 
excellent  regulation  of  unconsciousness  and  allowed  for  a  safe  and  effective  administration 
of  the  anesthetic  agent  propofol.  However,  the  adaptive  and  neuroadaptive  controllers  pre¬ 
sented  in  [49]  did  not  account  for  measurement  noise  in  the  EEG  signal.  Clinical  testing  has 
clearly  demonstrated  the  need  for  developing  adaptive  and  neuroadaptive  controllers  that 
can  address  system  measurement  noise  [49]. 

In  this  research  [37],  we  extend  the  neuroadaptive  controller  framework  developed  in  [7] 
to  address  measurement  noise  in  the  BIS  signal.  Specifically,  we  develop  an  output  feedback 
neural  network  adaptive  controller  that  operates  over  a  tapped  delay  line  (TDL)  of  available 
input  and  filtered  output  measurements.  The  neuroadaptive  laws  for  the  neural  network 
weights  are  constructed  using  a  linear  observer  for  the  nominal  normal  form  error  system 
dynamics.  The  proposed  framework  is  Lyapunov-based  and  guarantees  ultimate  boundness 
of  the  error  signals.  In  addition,  the  rionnegative  neuroadaptive  controller  guarantees  that 
the  physiological  system  states  remain  in  the  rionnegative  orthant  of  the  state  space.  Fi¬ 
nally,  we  present  numerical  and  clinical  results  for  controlling  the  depth  of  anesthesia  in  the 
operating  theater  during  surgery.  The  proposed  automated  anesthesia  neuroadaptive  con¬ 
troller  demonstrates  excellent  regulation  of  unconsciousness  and  allows  for  a  safe  and  effective 
administration  of  the  anesthetic  agent  propofol  in  the  face  of  noisy  EEG  measurements. 

2.19.  Direct  Adaptive  Control  for  a  Mutli-Compartmental  Model 
of  a  Pressure-Limited  Respirator  and  Lung  Mechanics  Sys¬ 
tem 

Mechanical  ventilation  of  a  patient  with  respiratory  failure  is  one  of  the  most  common 
life-saving  procedures  performed  in  the  intensive  care  unit.  However,  mechanical  ventilation 
is  physically  uncomfortable  due  to  the  noxious  interface  between  the  ventilator  and  patient 
and  mechanical  ventilation  evokes  substantial  anxiety  on  the  part  of  the  patient.  This  will 
often  be  manifested  by  the  patient  “fighting  the  ventilator.”  In  this  situation,  there  is 
dyssynchrony  between  the  ventilatory  effort  of  the  patient  and  the  ventilator.  The  patient 
will  attempt  to  exhale  at  the  time  the  ventilator  is  trying  to  expand  the  lungs  or  the  patient 
will  try  to  inhale  when  the  ventilator  is  decreasing  airway  pressure  to  allow  an  exlialatioh. 
When  patient- ventilator  dyssynchrony  occurs,  at  the  very  least  there  is  excessive  work  of 
breathing  with  subsequent  ventilatory  muscle  fatigue  and  in  the  worst  case,  elevated  airway 
pressures  that  can  actually  rupture  lung  tissue.  In  this  situation,  it  is  a  very  common 
clinical  practice  to  sedate  patients  to  minimize  “fighting  the  ventilator.”  Sedative-hypnotic 
agents  act  on  the  central  nervous  system  to  ameliorate  the  anxiety  and  discomfort  associated 
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with  mechanical  ventilation  and  facilitate  patient-ventilator  synchrony.  In  this  research,  we 
developed  an  adaptive  feedback  controller  for  alleviating  the  dyssynchrony. 

In  a  recent  paper  [56],  we  extended  the  existing  models  for  ventilation  systems,  to  obtain 
a  general  mathematical  model  for  the  dynamic  behavior  of  a  multi-compartment  respira¬ 
tory  system  in  response  to  an  arbitrary  applied  inspiratory  pressure.  Specifically,  we  used 
compartmental  dynamical  system  theory  to  model  and  analyze  the  dynamics  of  a  pressure- 
limited  respirator  and  lung  mechanics  system,  and  showed  that  the  periodic  orbit  generated 
by  this  system  is  globally  asymptotically  stable.  Furthermore,  we  showed  that  the  indi¬ 
vidual  compartinental  volumes,  and  hence  the  total  lung  volume,  converge  to  steady-state 
end-inspiratory  and  end-expiratory  values.  In  this  research  [73],  first,  we  develop  a  model 
reference  direct  adaptive  controller  framework  where  the  plant  and  reference  model  dynam¬ 
ics  are  switching  and  time-varying.  Next,  we  apply  the  proposed  adaptive  framework  to 
the  mult i-compart mental  model  of  a  pressure-limited  respirator  and  lung  mechanics  system. 
Specifically,  we  develop  an  adaptive  feedback  controller  that  stabilizes  a  given  limit  cycle 
corresponding  to  a  respiratory  pattern  identified  by  the  clinician  as  a  plausible  breathing 
pattern.  Finally,  we  provide  simulations  that  quantify  dyssynchrony  in  a  controlled  mechan¬ 
ical  ventilator  model. 
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4. 


Interactions  and  Transitions 


4.1.  Participation  and  Presentations 

The  following  conferences  were  attended  over  the  past  three  years. 

American  Control  Conference,  Minneapolis,  MN,  June  2006. 

IEEE  Conference  on  Decision  and  Control,  San  Diego,  CA,  December  2006. 

American  Control  Conference,  New  York,  NY,  July  2007. 

IEEE  Conference  on  Decision  and  Control,  New  Orleans,  LA,  December  2007. 

American  Control  Conference,  Seattle,  WA,  June  2008. 

IEEE  Conference  on  Decision  and  Control,  Cancun,  Mexico,  December  2008, 

Furthermore,  conference  articles  [44-74]  were  presented. 

4.2.  Transitions 

Our  work  on  adaptive  and  neuroadaptive  control  of  drug  delivery  partially  supported 
under  this  program  has  transitioned  to  clinical  studies  at  the  Northeast  Georgia  Medical 
Center  in  Gainesville,  Georgia,  under  the  direction  of  Dr.  James  M.  Bailey  (770-534-1312), 
director  of  cardiac  anesthesia  and  consultant  in  critical  care  medicine.  To  date,  we  have 
performed  over  forty  clinical  trials. 

In  critical  care  medicine  it  is  current  clinical  practice  to  administer  potent  drugs  that 
profoundly  influence  levels  of  consciousness,  respiratory,  and  cardiovascular  function  by  man¬ 
ual  control  based  on  the  clinician’s  experience  and  intuition.  Open-loop  control  by  clinical 
personnel  can  be  tedious,  imprecise,  tiine-consuming,  and  sometimes  of  poor  quality,  de¬ 
pending  on  the  skills  and  judgment  of  the  clinician.  Military  physicians  may  face  the  most 
demanding  of  critical  care  situations  when  dealing  with  the  causalities  of  hostile  action  and 
in  these  situations,  precise  control  of  the  dosing  of  drugs  with  potent  cardiovascular  and 
central  nervous  system  effects  is  critical. 

It  has  been  an  aphorism  among  anesthesiologists  since  World  War  II  that  “thiopental  (a 
common  drug  for  the  induction  of  anesthesia)  killed  more  Americans  at  Pearl  Harbor  than 
the  enemy,”  referring  to  the  consequences  of  cardiovascular  collapse  induced  by  thiopental 
in  trauma  patients.  Furthermore,  military  medicine  faces  unique  challenges  compared  to  the 
civilian  sector.  The  necessity  of  triage  has,  sadly,  been  a  not  rare  event  in  times  of  war  due  to 
unexpected  numbers  of  casualties  overwhelming  available  resources  and  furthermore,  health 
care  providers  may  be  among  the  casualties.  Because  of  the  possibility  of  demands  on  health 
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care  providers  that  may  exceed  local  resources,  we  believe  that  it  is  crucial  to  investigate 
the  use  of  advanced  control  technology  to  extend  the  capabilities  of  the  health  care  system 
to  handle  large  numbers  of  casualties.  Closed-loop  control  based  on  appropriate  dynamical 
system  models  can  improve  the  quality  of  drug  administration  in  surgery  and  the  intensive 
care  unit,  lessening  the  dependence  of  patient  outcome  on  the  skills  of  the  clinician. 

This  work  was  recently  communicated  to  Colonel  Leopoldo  C.  Cancio  (210-916-3301)  of 
the  US  Army  Institute  of  Surgical  Research  in  Fort  Sam  Houston,  San  Antonio,  in  order  to 
provide  improvements  for  combat  casualty  care  in  current  and  future  battlefields.  Transition 
discussions  are  ongoing. 
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